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Amendments to the Claims: 

A listing of the entire set of pending claims (including amendments to the 
claims, if any) is submitted herewith per 37 CFR 1.121. This listing of claims will 
replace all prior versions, and listings, of claims in the application. 

Listing of Claims: 

1 .(Currently Amended) A system (100) for integrative analysis of intrinsic (4^ 
and extrinsic (44)-audio-visual data, the system comprising: 

an intrinsic content analyser, the intrinsic content analyser being 
communicatively connected to an audio-visual source, the Intrinsic content analyser 
being adapted to search the audio-visual source for intrinsic data and being adapted 
to extract intrinsic data using an extraction algorithm, 

an extrinsic content analyser, the extrinsic content analyser being 
communicatively connected to an extrinsic information source, the extrinsic content 
analyser being adapted to search the extrinsic information source and being adapted 
to retrieve extrinsic data using a retrieval algorithm, wh e re i n th e i ntr i nsio - data - ar t d - th e 
oxtrins i o data ar e corr el ated, th e r e by a nd 

a processor configured to correlate the intrinsic data and the extrinsic data for 
providing a multisource data structur e, wherein the audio-visual source is a film and 
the extrinsic information source is a film screenplay written before production of the 
film. 

2. (Currently Amended) A -The s ystem according to claim 1, wherein the 
retrieval of the extrinsic data is based on the extracted intrinsic data. 

3. (Currently Amended) A -The s ystem according to claim 1, wherein the 
extraction and/or retrieval algorithm(s) is/are provided by a module. 
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4. {Current!y Amended) A- The system according to claim 1, wherein a query is 
provided by a user, the query being provided to the extraction algorithm and wherein 
the intrinsic data is extracted in accordance with the query. 

5. (Currently Amended) A -The s ystem according to claim 1 , wherein a query is 
provided by a user, the query being provided to the retrieval algorithm and wherein 
the extrinsic data is retrieved in accordance with the query. 

6. (Currently Amended) A -The s ystem according to claim 1 , wherein a feature 
reflected In the intrinsic and extrinsic data tf^otade -includes t extual, audio and/or 
visual features. 

7. (Currently Amended) A- The system according to claim 1 , wh e r ei n th e 
audio visual source i s a fi l m (101) and w herein the extracted data include textual 
f4G4>, audio and/or visual features (1 05, 106) . 

8. (Currently Amended) A -The system according to claim 1, wherein the 
extrinsic information source is connected to and may be accessed via the Internet 

r±fvi\ 

Claim 9 (Canceled) 

1 0. (Currently Amended) A -The system according to c l aim 9 claim 1 , wherein 
the extrinsic content analyser include knowledge about screenplay grammar, and 
wherein the extrinsic data is retrieved based on information extracted from the 
screenplay by use of the screenplay grammar. 

1 1 .(Cun'ently Amended) A -The s ystem according to c l aim 9 claim 1 . w herein 
the identification (5) of persons in a film is obtained by means of the screenplay. 
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12. (Currently Amended) A- The system according to claim 9 claim 1 . w herein 
a feature in a film is analysed based on information included in the screenplay. 

13. (Currently Amended) A -The s ystem according to claim 1 , wherein the 
correlation of the intrinsic and extrinsic data is time correlation4434), thereby 
providing a multisource data structure where a feature reflected in the intrinsic data is 
time correlated to a feature reflected in the extrinsic data. 

14. (Currently Amended) A -The s ystem according to claim 13, wherein the 
time correlation is obtained by an alignment of a dialogue (120) in the screenplay to 
the spoken text (104) in the film and thereby providing a timestamped transcript 
(454fof the film. 

16.(Currently Amended) A -The system according to claim 14, wherein a 
speal^er identification in the film is obtained from the timestamped transcript. 

16. (Cunrently Amended) A -The system according to cla i m 9 claim 1 . wherein 
the screenplay is compared with the spoken text in the film by means of a self- 
similarity matrix-(3Q). 

17. (Currently Amended) A -The s ystem according to claim 1 , wherein a high- 
level infomiation structure (5 9 ) is generated in accordance with the multisource data 
structure. 

1 8. (Currently Amended) A- Jhe system according to claim 1 7, wherein the 
high-level information structure may be stored on a storage medium. 

19. (Currently Amended) A -The system according to claim 17, wherein an 
updated high-level infonnation structure Is generated, the updated high-level 
information structure being an already existing high-level information structure which 
is updated in accordance with the multisource data structure. 
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20. (Currently Amended) A -The system according to claim 1, wherein the 
retrieval algorithm is a dynamic retrieval algorithm adapted to dynamically update 
itself by including additional functionalities in accordance with retrieved extrinsic data. 

21. (Currently Amended) A -The system according to claim 20, wherein the 
additional functionaiities is obtained by training the retrieval algorithm on a set of 
features from intrinsic data using labels obtained from the extrinsic data. 

22. (Currently Amended) A -The s ystem according to-6laim- & claim 1 . wherein 
the training is performed using at least one screenplay. 

23. (Currently Amended) A- The system according to 1, wherein an automatic 
ground truth identification in a film is obtained based on the multisource data 
structure for use in benchmarl^ing algorithms on audio-visual content. 

24. (Currently Amended) A -The system according to 1, wherein an automatic 
scene content understanding in a film is obtained based on the textual description in 
the screenplay and the audio-visual features from the film content. 

25. (Currently Amended) A -The s ystem according to 1, wherein an automatic 
labelling in a film is obtained based on the multisource data structure. 

26. (Currently Amended) A method for integrative analysis of intrinsic and 
extrinsic audio-visual information, the method comprising the steps -acts of: 

searching an audio-visual source for intrinsic data and extracting intrinsic data 
using an extraction algoiithm, 

searching an extrinsic information source and retrieving extrinsic data using a 
retrieval algorithm, and 
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correlating the intrinsic data and extrinsic dat a , th e r e by data, for p roviding a 
multisource data structure , wherein the audio-visual source is a film and the extrinsic 
information source is a film screenplavwrttten before production of the film . 

27. (Currently Amended) A -The method according to claim 26 further 
comprising the step of generating a high-level information structure in accordance 
with the multisource data structure. 

28. (Currently Amended) A -The method according to claim 26, wherein the 
extrinsic content analyser Include knowledge about screenplay grammar, and 
wherein the extrinsic data is retrieved using information extracted from the 
screenplay by use of the screenplay grammar. 

29. (Currently Amended) A -The method according to claim 26, wherein the 
retrieval algorithm is updated by training the algorithm on a set of extrinsic data. 

Claims 30-32 (canceled) 
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